In recent years, so-called electronic publishing has been proposed as a technique for issuing publications. Electronic publications published by the electronic publishing comprise sentence data representing sentences and image data representing images such as illustrations, and are memorized on memory media. A reader prepares a device provided with a memory medium reading device and a display device in advance, and loads the memory medium into the reading device. The image data and the sentence data on the memory medium are read by the reading device and supplied to the display device. The image represented by the image data is displayed on the display screen of the display device. The sentence data is first converted into image data by using character fonts provided in the display device in advance, and the image represented by the image data after the conversion is displayed on the display screen of the display device.
A liquid crystal display device and a cathode ray tube are taken as examples of the display device. The display screen of the display device comprises a plurality of display pixels arranged in matrix. The image comprises a plurality of display pixels arranged in matrix. The image data comprises pixel data representing the luminance of each of pixels constituting the image. The display device illuminates each display pixel at the luminance depending on the pixel data. As a result, the image represented by the image data is displayed on the display screen.
At present, many electronic publications are data forms of published books, that is, contents. Therefore, in the case of generating the electronic publications, the sentence data and the image data are frequently generated by using the contents. In order to generate the electronic publications by using the contents, first, each page of the contents is separated into a portion on which sentences are printed and a portion on which pictures are printed. Next, the portion on which pictures are printed is read by a scanner whereby the image data is generated. Then, the portion on which sentences are printed is read by the scanner whereby image data is generated, and the image data is subjected to a character recognition process whereby the sentence data is generated.
In the case of generating the sentence data in accordance with the procedure, wrong characters and missing characters may occur in the sentences represented by the sentence data owing to recognition errors and the like during the character recognition process. Therefore, it is necessary to proofread the sentences represented by the sentence data once again at the step of generating the sentence data, although the sentences have already been proofread at the step of generating the contents. As a result, the sentences are proofread twice, whereby the generation of the sentence data takes time and labor, and the cost of generating the sentence data increases.
In addition, since the sentence data is the so-called text data for example, each character constituting the sentences is represented by a character code in the sentence data. Therefore, in the case of displaying the sentences represented by the sentence data on the display device, each character of the sentences is represented by a font provided in the display device. For this reason, the font used in the case of displaying the character on the display device may differ from the font used in the case of printing the character on each page of the contents. As a result, the impression of the appearance of the sentences displayed on the display device may differ from that of the sentences printed on each page of the contents, and this difference may make readers feel uncomfortable.
In order to save time and labor for proofreading and to maintain the impression of the appearance of the sentences printed on each page of the contents, it is considered that each page of the contents is not separated into sentences and pictures, but it is assumed to be a single picture. In this case, an electronic publication comprises image data representing each entire page of the contents, and the image data is generated by reading each page of the contents by using the scanner. At this time, the following three problems occur.
As a first problem, in the case of generating the image data by reading each page of the contents by using the scanner, the contrast of the image represented by the image data is not necessarily the maximum contrast in conformity with the standard of the image data. This occurs because the color of the background of each page of the contents is not necessarily true white, and because the color of the ink is not necessarily true black. In addition, this also occurs owing to the characteristics of the scanner. In the case where an image with a low contrast is displayed on the display device, it may be difficult to read characters. In the case where the display device itself has a low contrast, the visibility of the characters in the image displayed on the display screen is likely to become lower in comparison with the image having the maximum contrast in conformity with the standard of the image data. The image having a low contrast is an image wherein the color of the pixels at the background portion in the image is a color other than true white, such as cream, and the color of the pixels constituting the characters in the image is a color other than true black, such as dark brown as shown in FIG. 23A. The image having the maximum contrast is an image wherein the color of the pixels at the background portion in the image is true white and the color of the pixels constituting the characters in the image is true black as shown in FIG. 23B.
Japanese Unexamined Patent Publication JP-A 63-39280 (1988) has proposed an image processing device for raising the contrast of an image by using a gamma correction process in the case where the gradation of the image is the so-called halftone, that is, when the gradation concentrates on gray, that is, when the contrast of the image is low. In the image processing device in accordance with the present publication, gamma conversion tables individually corresponding to the gradation distributions of a plurality of images have been memorized in a RAM beforehand. When an image to be processed is supplied, the image processing device first detects the gradation distribution of the image and reads one of the gamma conversion tables corresponding to the detected graduation distribution from the RAM. By using the gamma conversion table having been read, the image is subjected to gamma correction. In the image processing device of the present publication, the image processing device of the present publication estimates the gradation distribution of an image to be processed in advance, and the gamma conversion table is generated in accordance with the estimated gradation distribution. Therefore, in the case of processing an image having an unestimated gradation distribution, any gamma table corresponding to the gradation distribution of the image is not available, whereby it is difficult to raise the contrast of the image.
As a second problem, the visibility of an image displayed on the display screen of the display device lowers owing to the fact that the resolution of image data does not coincide with the resolution of the display device. This second problem will be explained below. In the display screen of the display device, generally speaking, a plurality of display pixels are arranged in matrix as shown in FIG. 24, and its resolution differs depending on the number of the display pixels. FIG. 24A is a schematic view showing the whole of the display screen 1, and FIG. 24B is a magnified schematic view showing the portion 2 of the display screen 1. In the following drawings, a pixel and a display pixel are respectively represented by a square, and the luminance of the pixel and the display pixel is represented by the number of oblique lines in the square. The higher the luminance of the pixel and the display pixel, the fewer the number of the oblique lines.
Generally, the resolution of the display device is lower than the resolution of the scanner and the resolution of the printing of the contents. Therefore, small characters formed of dots and lines smaller than the size of the display pixel have been printed as the contents. The small characters cannot be displayed basically on the display screen; however, in the case where the so-called monochrome halftone is used, they can be displayed in a pseudo way. In order to display the small characters by using the monochrome halftone, the luminance of each display pixel is set at the mean value of the luminance of the background in the portion in the contents corresponding to each display pixel and the luminance of a character portion in the portion.
In the case where the small characters are displayed by using the monochrome halftone, a person who watches the display screen feels that the small characters are smoothly displayed, without being conscious of the display pixels, and receives the impression that the small dots and lines are blurred. This is because of the following reason. As shown in FIG. 25A, in the case where the width of the line segments constituting a character 4 drawn in the contents 3 and the size of dots constituting the character 4 are not larger than the size of a display pixel 5, each luminance value of the plurality of display pixels 5 including at least portions of the line segments and dots is determined depending on the ratio of the area of each display pixel 5 and the least portions. In other words, in this case, the luminance of the least portion is distributed to the whole of the display pixel 5. Therefore, the luminance of the display pixel 5 becomes lowered than the luminance of the least portion as shown in FIG. 25B. Accordingly, the small characters give blurred impression.
The image giving blurred impression because of the reason can be converted into an easy-to-see image without blurs by carrying out a sharpening process using the so-called Laplacian. Japanese Unexamined Patent Publications JP-A 5-167852 (1993) and JP-A 7-240841 (1995) are available as conventional technologies regarding the sharpening process.
JP-A 5-167852 has, in the case of carrying out the sharpening process for an image, proposed an image sharpening method for preventing the portions having flat luminance changes in the image from becoming coarse. In the case of carrying out the image sharpening process, first, the second-order differentiation value of the luminance of each of pixels of the image to be processed is obtained as the sharpening evaluation function of each pixel. Next, a coefficient representing the sharpening level of each pixel is determined depending on the sharpening evaluation function of each of the pixels. The sharpening process using the coefficient is carried out for each of the pixels.
In the case of paying attention to the edges of characters in the image and carrying out the sharpening process to enhance the sharpening levels of the edgy portions, the small characters themselves in the image may collapse and the their edges are less likely to be intensified, whereby the sharpening levels of the characters portions in the image are likely to become low. In addition, in the case, the edges of large characters or line segments are intensified at their edges; however, if the sharpening levels of the large characters and line segments are intensified, the line segments become unsmooth and have the so-called jaggies conspicuously. Therefore, in the case of using the sharpening process, it is difficult to make small characters easy to read.
In addition, JP-A 7-240841 has proposed an image sharpening processing device for obtaining the same processing result in the case of subjecting an image to the sharpening process, regardless of the image deteriorating characteristics of the scanner used to generate image data. First, the image sharpening processing device calculates a coefficient for representing the sharpening level of the sharpening process by using scanner characteristic parameters determined on the basis of the MTF of the scanner. Next, the sharpening process using the coefficient is carried out for the image to be processed.
For example, when the image shown in FIG. 26A is subjected to the sharpening process described in the publication, the image shown in FIG. 26B is obtained. If the level of the sharpening is excessive in this case, the luminance distribution of the image becomes close to that of a binary image, whereby the smoothness of characters displayed by using the monochrome halftone is lost. Therefore, in the case of using the sharpening process to carry out uniform sharpening for the whole of an image without considering characters and pictures as explained in the publication, it is difficult to make the small characters easy to read.
Furthermore, Japanese Unexamined Patent Publication JP-A 6-308924 (1994) has proposed a display device for clearly defining the boundary between two different color portions in an image in the case where the image is displayed on a display device. The display screen of the display device comprises a plurality of dots arranged in matrix, and the color of each dot is determined by a plurality of data items in image data representing the image. In the case where data for determining the colors of a plurality of dots constituting a given row or a given column on the display screen is arranged in the same order as that of the dots, if a plurality of data items for dots having the same color are arranged, the data at the most end of the plurality of data items is converted into data for determining the color of dots to black. In the case where the process is used for the image data constituting the electronic publication, the pixel data for determining the color of pixels used to represent the small characters is scarcely converted into data for determining the color of dots to black, since the pixel data is not so much different in luminance from the pixel data for determining the color of the pixels around the small characters because of luminance distribution. Therefore, it is difficult to make the small characters easy to read.
A third problem is that the visibility of an image displayed on the display screen lowers because of the biased gradation characteristics of the display device. This will be described below specifically. The gradation characteristics of the display device are defined as the relationship between the luminance represented by pixel data and the luminance of the display pixels, and more specifically, represent how the luminance of the display pixels illuminated depending on the pixel data changes as the luminance represented by the pixel data changes. The gradation characteristics of the display device are generally nonlinear in many cases.
The gradation characteristics are represented by a gradation characteristic curve 11 in a graph in which the abscissa is the luminance represented by pixel data and the ordinate is the luminance of the display pixels as shown in FIG. 27 for example. The gradation characteristics are better as the gradation characteristic curve 11 is closer to a reference straight line 12 passing through the origin and having an inclination of 45 degrees. The gradation characteristic curve 11 of FIG. 27 is obtained by plotting the luminance of the plurality of display pixels on the imaginary line segment 13 of the image of FIG. 28 displayed on the display device and a plurality of pixel data items for respectively determining the luminance of the pixels. The pixel data is incremented sequentially by a predetermined value in the left-to-right arrangement order of the display pixels.
To correct the gradation characteristics, the image processing device including the display device is provided with a luminance correction table depending on the gradation characteristics. The luminance conversion curve 14 shown in FIG. 29 represents the relationship between the input luminance and the output luminance of the luminance correction table depending on the gradation characteristics represented by the gradation characteristic curve 11 of FIG. 27. In the case where an image to be displayed is subjected to a gradation correction process, the luminance of each pixel of the image is replaced with the output luminance in the luminance correction table corresponding to the input luminance in the luminance correction table, being equal to the luminance of each of the pixels. In the case where the image subjected to the gradation correction process is displayed on the display device, the curve 15 of FIG. 30 represents the relationship between the luminance of the image and the luminance of the display pixels of the display device. As shown in the graph of FIG. 30, in the case, the curve 15 coincides with the reference straight line 12.
Furthermore, for the gradation characteristic correction, the image processing device is provided with a gamma correction table corresponding to the gradation characteristics of the display device. The luminance conversion curve 16 in the graph of FIG. 31 represents the relationship between the input luminance and the output luminance in the gamma correction table corresponding to the gradation characteristics represented by the gradation characteristic curve 11 of FIG. 27. In the case where the image is subjected to the gamma correction process, the luminance of each pixel of the image is replaced with the output luminance in the gamma correction table corresponding to the input luminance values in the gamma correction table, being equal to the luminance of each of the pixels. In the case where the image subjected to the gamma correction process is displayed on the display device, the curve 17 of FIG. 32 represents the relationship between the luminance of the image and the luminance of the display pixels of the display device. As shown in the graph of FIG. 32, in the case, the curve 17 nearly coincides with the reference straight line 12.
In the case where the gradation characteristics of the display device are biased, that is, nonlinear, the image displayed on the display screen becomes harder to see as the gradation characteristic curve 11 is more remote from the reference straight line 12. The change in the visibility of an image depending on the bias of the gradation characteristics is inconspicuous in the case when the image is the so-called gray-scale image, that is a picture; however, the change becomes conspicuous in the case where the small characters are drawn in the image. In the latter case, as the gradation characteristics are more biased, the balance of the ratio between the black portion and the white portion in the region in the image displayed on the display screen, in which the small characters are described, appears to be lost from its original balance. For example, if the image shown in FIG. 33A is subjected to the gradation correction process, the image shown in FIG. 33B is obtained. In this way, a plurality of pixels which should have the same density to represent lines having the same thickness may become faint or dense partially in the region. Therefore, the characters in the region become hard to see because of unevenness. In particular, in the case where the display device is a liquid crystal display device, white small regions are likely to collapse in general; and when the image is displayed, the small characters fade in many cases.
As described above, the image processing device has the three problems. In addition, even when the conventional technologies proposed in the publications are applied to the image processing device, it is difficult to solve the first and second problems. Furthermore, a display device, the bias of the gradation characteristics of which can be adjusted by a user as desired to solve the third problem, is rarely available; for this reason, it is further difficult to solve the third problem.
An object of the invention is to provide an image processing device and an image processing method capable of preventing the reduction of the visibility of an image owing to the bias of the gradation characteristics, the contrast of the image and the resolution of the image.